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Abstract — We derive a new achievable rate region for the 
problem of communicating over a multiple access channel with 
states. Our coding technique is based on the ensemble of nested 
coset codes and the technique of typicality decoding. Exploiting 
structure in this ensemble, we analyze a more efficient decoding 
strategy to improve upon the rate region achievable using 
unstructured random codes. We identify examples for which the 
achievable rate region based on nested coset codes are strictly 
larger than the ones achievable using random unstructured codes. 



I. Introduction 

The technique of proving achievability of rate regions 
in information theory is random coding. Traditionally, the 
distribution induced on the ensemble of codes is such that 
individual codewords are mutually independent. Furthermore, 
in communication models with multiple terminals, the differ- 
ent codebooks are mutually independent of each other Such 
an analysis has proved sufficient/optimal for single user and 
particular multi terminal communication problems 

The problem of distributed reconstruction of modulo-2 sum 
of binary sources studied by Korner and Marlon ||T1 proved to 
be the first exception. The authors proposed a coding technique 
based on nested linear codes that strictly outperforms the 
best known strategy based on independent unstructured codes. 
More recently, a similar phenomenon has been identified by 
Philosof and Zamir |2 | for a particular multiple access channel 
with channel state information distributed at the transmitters 
(MAC-DSTx). Restricting their attention to a binary doubly 
dirty MAC-DSTx (BDD-MAC), the authors propose a coding 
technique, henceforth referred to as PZ-technique, based on 
nested linear codes that is optimal and moreover, strictly more 
efficient than the best known strategy based on unstructured 
independent codes. 

Nevertheless ingenious, PZ-technique f2] is particular to the 
additive nature of the BDD-MAC studied therein. This tech- 
nique being strictly more efficient than the currently known 
best strategy based on independent unstructured codes raises 
the following question. Can the PZ-technique be generalized 
for communicating over an arbitrary discrete MAC-DSTx, and 
if yes, would this yield an achievable rate region strictly larger 
than the best known achievable rate region using unstructured 
independent codes even for non-additive MAC-DSTx? 

Arun Padakandla and Sandeep Pradhan are with the Department of Elec- 
trical and Computer Engineering, University of Michigan, Ann Arbor 48109- 
2122, USA. 

'However, characterization of optimal performance in many multi-terminal 
communication problems such as distributed source coding, interference 
channel, broadcast channel, multiple description coding remain open. 



In this article, we provide affirmative answers to the above 
questions. We begin by identifying two central elements of 
PZ-technique 1) decoding mod— 2 sum, instead of the pair, 
of auxiliary codewords and 2) choosing source code of each 
user's code to be cosets of a common linear code to enable 
containment of the range of this mod— 2 sum of auxiliary 
codewords. In the first step of our generalization, presented 
we propose the use of nested coset codes f3\ 



III 



in section 

and analyze decoding the sum of auxiliary codewords over 
an arbitrary MAC-DSTx. The key element in this step is the 
use of joint typical encoding and decoding that enables us 
analyze the probability of error over an arbitrary MAC-DSTx. 
Indeed, the analysis of joint typical encoding and decoding of 
correlated codebooks with statistically dependent codewords 
involves several new elements. The reader is encouraged to 
peruse these in proof of theorem |3] 

The significance of the rate region proved achiev able in t he 
first step is illustrated through examples in section 



III-C 



In 

particular, we provide examples of MAC-DSTx that are not 
additive, but yet benefit from decoding the sum of auxiliary 
codewords chosen from nested coset codes. The channels 
being non-additive, it is significantly harder to provide ana- 
lytical upper bounds and we resort to computing rate regions 
achievable using unstructured independent and nested coset 
codes. These examples illustrate that structured code based 
strategies do not hinge on the channel being additive but 
would benefit as long as the optimizing test channel from the 
auxiliary inputs to the channel output is not far from additive. 

Does the rate region proved achievable using nested coset 
codes subsume the largest known achievable rate region using 
unstructured independent codes? It is our belief that strategies 
based on structured codes are not in lieu of their counterparts 
based on unstructured codes. In most cases, structured codes 
enable efficient decoding of a 'compressive]^ function. The 
penalty payed in rate for imposing structure in the code is 
compensated for by the compressive nature of the function. 
However, for decoding the pair of codewords, structured codes 

^The coding technique proposed in the first step reduces to that proposed 
in f2l for BDD-MAC and moreover Philosof and Zamir have proved strict 
sub-optimality of unstructured independent coding for this BDD-MAC. This 
in itself establishes significance of theorem|3] Notwithstanding this, it is easy 
to argue significance of our generalization by appealing to continuity. An 
additive channel can be perturbed slightly to result in a non-additive channel 
for which the technique proposed in |2| are not applicable. By continuity 
of the rate regions as a function of the channel parameters, one can see 
why a generalization of | 2| must perform strictly better than unstructured 
independent coding. The examples presented in section [Tll-C [ corroborate this. 

^ f{Ui,U2) is 'compressive' if H{f(Ui,U2)) is significantly lower than 
H{Ui,U2). 



are not optimal. Indeed, Korner Marton strategy is outper 
formed by Berger-Tung strategy for the class of distributions 
for which the modulo— 2 sum is not sufficiently compressivd^ 
We therefore take the approach of Ahlswede and Han ||4 
Section VI], where in structured and unstructured coding 



techniques are glued together. In section IV we derive the 



largest known achievable rate region for communicating over a 
MAC-DSTx that combines structured and unstructured coding 
techniques. 

If the channel is far from additive, it may not be efficient to 
decode the sum of auxiliary inputs. For example, if the input, 
source and output alphabets are a group and MAC-DSTx is 
doubly dirty with the operation being the group multiplication, 
then it is natural to decode the product of auxiliary inputs. In 
other words, the technique of decoding sum of codewords as 
proposed in 121 must be generalized to decoding any arbitrary 
bivariate function of the auxiliary inputs. The task of building 
codes that are closed under other bivariate functions and their 
analysis using joint typical encoding and decoding rules is 
quite involved and we only provide pointers in section |V] and 
illustrate the gains achievable using group theoretic examples. 

Our findings are motivated by a belief that 1) strategies 
based on independent unstructured codebooks are inadequate 
and 2) codebooks endowed with algebraic structure may 
be employed to develop and analyze more efficient coding 
techniques for multi-terminal communication. Indeed, even 
after three decades, since the findings of Korner and Marton 
came to light, we are yet unaware of a coding technique based 
on independent unstructured codes that achieve rates promised 
in On a positive note, recent findings [5|, [6|, |7| lend 
sufficient credence for the need of structured codes. Sridharan 
et. al. fSl propose a coding technique based on lattices for 
communicating over a A'— user Gaussian interference channel 
(K > 3) that strictly outperforms a natural extension of Han- 
Kobayashi technique |9|. We propose an analogous coding 
technique based on nested linear codes |,10J for the general 
discrete 3— user interference channel and identify an example 
for which the proposed technique outperforms the natural 
extension of Han-Kobayashi technique [9|. Krithivasan and 
Pradhan flTl propose a framework based on structured codes 
for the problem of distributed source coding that outperforms 
the best known strategy based on structured codes due to 
Berger-Tung lfT2ll . We have employed the same ensemble of 
nested coset codes to strictly enlarge the largest known achiev- 
able rate regiorj^ for the general 3— user discrete broadcast 
channel in |fT3l . 

We summarize by stating the significance of our contri- 
bution. Firstly, we enlarge the largest known achievable rate 
region for a general discrete MAC-DSTx using nested coset 
codes. In addition to the BDD-MAC studied by Philosof 



H(X,Y) 



then it is better to reconstract X ffi Y using 



''If H{X OY)> 2 
the technique of Berger-Tung. 

""This is obtained by a natural extension of Marton's coding technique 
proposed for 2— user broadcast channel. Our findings therefore suggest strict 
sub-optimality of Marton's coding technique, that has been the largest known 
for over three decades, for 3 or more users. 



and Zamir, we present non-additive examples for which the 
proposed rate region strictly enlarges the currently known 
largest achievable rate region using unstructured independent 
codesj^ Secondly, and perhaps more importantly, our findings 
hint at a general theory of structured codes. Linear and nested 
linear codes have been employed to derive communication 
strategies for particular additive source and channel coding 
problems that outperform all classical/unstructured known 
techniques. However the question remains whether these struc- 
tured code based techniques can be generalized to arbitrary 
multi-terminal communication problems. Our findings indicate 
that strategies based on structured codes can be employed to 
analyze more intelligent encoding and decoding techniques 
for an arbitrary multi-terminal communication problem. This 
opens up the possibility of exploiting new degrees of freedom 
to enlarge achievable rate regions for many multi-terminal 
communication problems that have resisted a solution. Finally, 
the analysis of typical set decoding of correlated codebooks 
with statistically dependent codebooks yields new elements in 
the proof technique. 

II. Preliminaries and problem statement 

A. Notation 

We employ notation that is now widely adopted in the 
information theory literature supplemented by the following. 
We let denote a finite field of cardinality q. While + 
denotes addition in M, we let denote addition in a finite 
field. The particular finite field, which is uniquely determined 
(up to an isomorphism) by it's cardinality, is clear from 
context. When ambiguous, or to enhance clarity, we specify 
addition in using (Bq. For elements a,b, in a finite field, 
a Q b : — a ® {—b), where (—6) is the additive inverse of b. 
The log and exp functions are taken with respect to the same 
base. For concreteness, the base may be assumed to be 2, in 
which case, units for information theoretic quantities such as 
entropy and mutual information would be bits. If f -.U ^ X 
is a map, the n-letter extension of / denoted /" : U" 
is defined /" (u") := (/ {ui) : i = 1, 2, • • • , n). 

B. MAC-DSTx 

Consider the two user multiple access analogue of the point 
to point channel with state (PTP-STx) studied by Gelfand and 
Pinsker f\A\. Let Xi and X2 denote finite input alphabet sets 
and y, the output alphabet set. Transition probabilities depend 
on a random parameter S : = {81,82), called state, that 
takes values in a finite set 5 : = 5i x ^2. The discrete time 
channel is time invariant, memoryless, used without feedback. 
Let VFy|xs(j/l^> ■s) be probability of observing y E y given 
X : = {xi,X2) e A" : = Afi X A2 is input to the channel in state 
s : — (si,S2) €E S. The state at time i. Si is (i) independent 
of {St, Xt,Yt) : 1 < t < i, and (ii) identically distributed for 
all i. Let Ws{s) be probabiUty of MAC-DSTx being in state 
s E S. We assume 5" is non causally known to encoder j. In- 
put Xj is constrained with respect to an additive cost function 

^This is based on computing the largest known achievable rate region using 
unstructured independent codes. 



Kj : Xj X Sj [0,00). We refer to this channel as MAC- 
DSTx (5, Ws,X,y, Wy\x.s-, «)■ The subject of this article is 
the information theoretic capacity region of MAC-DSTx and 
in particular, an achievable rate region. We refer the reader to 
II2I for standard definitions of a code, achievability, capacity. 
Let C(t) : = cl{R = (i?i,i?2) e : (R.,-r) is achievable} 
denote the capacity region when constrained to average cost 

of T = (ti,T2). 

C. Largest known achievable rate region 

The coding technique that achieves capacity of PTP-STx 
lfT4l can be generalized to obtain the largest known inner 
bound to C{t). We provide a characterization of the same 
below. 

Definition 1: Let D(x) be collection of distributions 
PuxsY onU^ X X X S X y, where U denotes Ui, U2, such 
that (i) ps = Ws, (ii) Py\xsu = Py\xs = WV|xs. 0") 
PUj\s = Pu,\s,^ (iv) Px,\su ^ Px,\SjUj for i = 1,2 (v) 
Px,\s,u,(xj\sj,Uj) e {0,1} for all {uj,s.j,Xj),j = 1,2 
and (vi) E{Kj{Xj,Sj)} < r,. For puxsY e D(t), let 
a{puxsY) be defined as the set 

iRi,R2) : Ri < I{U,;Y,U2) - I{Ui;Si), 
R2<IiU2;YUi)-IiU2;S2), 
Ri+R2< I{U: Y) + I{Ui; U2) - Ej=i HU,; Sj] 

and a{T) be convex closure of iip^^sYeniT)a{puxsY)- 
Theorem 1: q;(t) C C(t). 

Achievability of a(T) can be proved by employing the 
encoding technique proposed by Gelfand and Pinsker {\A\ at 
each encoder and joint decoding proposed by Ahlswede |15|, 
Liao lfT6l . In the sequel, we provide an illustration of this 
coding technique for BDD-MAC. 

D. Rate region achievable using unstructured codes for BDD- 
MAC 

Philosof and Zamir prove ^(t) C C(t) for BDD-MAC. In 
order to identify the key elements of PZ-technique, we briefly 
analyze unstructured coding (this section), PZ-technique (sec- 
tion 



III-A I and set the stage for a generalization. 



BDD-MAC is a MAC-DSTx with binary alphabets S.j 



Xj ~ y ^ {0,1}' j = 1,2. The state sequences are 
independent Bernoulli-1 processes, i.e., Ws{s) = \ for all 
s e 5. The channel transition is described by the relation 
Y = Xi ®2 Si 02 X2 ®2 'S'2- An additive Hamming cost is 
assumed on the input, i.e., Kj(l, Sj) ~ 1 and Kj(0, Sj) — for 
any Sj E Sj, j = 1,2 and the input is subject to a symmetric 
cost constraint r = (t, r). 

We describe the test channel pusxY G D(t) that 
achieves a{T). For each user j, consider the test channel 
that achieves the Gelfand-Pinsker capacity treating the other 
user as noise i.e., pc/,s,x, (0, 1, 1) = pu^s,Xk{^,0,l) = 
i,Pc/,s,;f, (0,0,0) = pu,s,x,{^A,0) = Philosof and 
Zamir prove pusx = PU1S1X1PU2S2X2 achieves a{T) = 
{R:Ri + R2<2hbiT)-l}. 

Let us take a closer look at achievability of the vertex 
{2hb{T) — 1,0) using the above test channel. Since user 2 has 



no message to transmit, it picks a single bin with 2"^('^2;S2) ^ 
2n(i-hb(T)) codewords independently and uniformly from the 
entire space of binary vectors. User 1 picks 2"^^ bins each 
with 2"^(^i'^i) = 2"(i~''''(^» independendy and uniformly 
distributed binary vectors. Encoder 2 observes S'2 and chooses 
a codeword, say U2, that is within a Hamming distance of 



from 55' and transmits 



^2" = 



®2 S"!^. Encoder 



1 



performs a similar encoding, except that it restricts the choice 
of [/" to the bin indexed by user I's message, and transmits 

= JJf 02 5*1". 

What is the maximum rate i?i at which user 1 can transmit 
it's message? Decoder receives = J7f ©2 U2 and looks 
for all pairs of codewords that are jointly typical with F". 
Since any pair of binary n— length vectors are jointly typical 
{Ui and U2 are independent and uniform), the decoding rule 
reduces to finding all pairs of binary n— length vectors in the 
pair of codebooks, that sum to the received vector F". All bins 
chosen independently without structure imply that any bin of 
user I's codebook when added to the user 2's codebook (a 
single bin) results in 2"(^^^'''''^'^^' distinct vectors. Therefore, 
we cannot hope to pack more than ^,.(2-2fefc(9)) = 2"'^^'''''''^"-'^^ 
bins in user I's codebook. 

We make a few observations. Effectively, communication 
occurs over the {Ui,U2) — Y channel and the test channel 
induces the Markov chain {Ui,U2) - Ui ®2 U2 - Y. It 
would therefore be more efficient to communicate information 
over the Ui ®2 U2 ~ Y channel which suggests an efficient 
utilization of Ui ©2 C/2— space. Having chosen codewords in 
each bin independently and moreover the two users' bins 
independently, each message pair utilizes 2"^^~^''''('^)) vectors 
in the Ui ©2 C/2— space. In section III-A we present PZ- 
technique, wherein structure in the codebooks is exploited for 
more efficient utilization of Ui ©2 C/2— space. 

We remark that an explosion in the range of sum of 
transmitted codewords severely limits achievable rate. In the 
next section, we study how codebooks endowed with structure 
can contain this explosion. 

III. An achievable rate region using nested coset 

CODES 

A. Nested linear codes for BDD-MAC 

We present PZ-technique proposed for BDD-MAC. The 
encoding and decoding techniques are similar to that stated in 
I II-D| except for one key difference. The bins of user 1 and 2's 
codebooks are cosets of a common linear code. In particular, 
let A/ denote a linear code of rate 1 — /ib(T) that can quantize 
a uniform source, state S*" in our case, within a Hamming 
distance of nr. Since user 2 has no message to transmit, it 
employs A/ as it's only bin. Encoder 1 employs 2"^^ cosets 
of A/ within a larger linear code, called Xq, as it's bins. Note 
that rate of Xq is i?i + l — /if,(T). Encoding rule is as described 
in section ITl-DI 

The codebook of user 2 when added to any bin of user I's 
code results in a coset of A/, and therefore contains at most 
2n(i-/i6(T)) codewords. Moreover, since [/" lies in A/, user 
2's codeword U2 and the received vector Y" = f/" ©2 U2 



lies in the same cosetQ Since the channel is noiseless user 1 
may employ all cosets of A/ and therefore communicate at 
rate hi,{T) which is larger than 2/it(r) — 1 for all r e (0, 1). 

Let us identify key elements of PZ- technique. Each message 
pair corresponds to at most 2"^^"''''^'^)) vectors in Ui ®2 
[/2— space, resulting in a more efficient utilization of this 
space. This difference is indeed the difference in the sum 
rate achievable using independent unstructured codes and PZ- 
technique. We also note the decoder does not attempt to disam- 
biguate the pair (L/", U2) and restricts to decoding Ui®2U2- 
This is motivated by the Markov chain (Ui, U2) — Ui(B2U2 — Y 
induced by the test channel and the use of structured code- 
books that contain the sum. 

It is instructive to investigate the efficacy of this technique 
if users 1 and 2 employ distinct linear codes A/i,A/2 of 
rate 1 — /ih(r) instead of a common linear code A/. In this 
case, each message of user 1 can result in 2^^2''''('^) received 
vectors which restricts user I's rate to 2hi,{q) — 1 and provides 
no improvement over the unstructured coding technique. We 
conclude that if the source codes of the MAC channel code are 
nontrivial, as in this case due to the presence of a state, then 
it maybe beneficial to endow the source code with algebraic 
structure that contains the range of a bivariate function, and 
enable the decoder decode this function of chosen codewords. 

Our generalization is based on the above principles. In the 



first step (section III-B 1 of our generalization, we decode the 
sum of chosen codewords over an arbitrary MAC using nested 
coset codes. 

B. An achievable rate region for arbitrary MAC-DSTx using 
nested coset codes 

In this section, we present the first step of our generalization 
- joint typical decoding of sum of auxiliary codewords using 
nested coset codes over an arbitrary MAC-DSTxj^ Indeed, the 
interplay of joint typical encoding and decoding and correlated 
codebooks result in new elements. The reader is encouraged 
to peruse details of the proof of theorem [3] in particular that 
of lemma |2] 

We begin with a characterization of test channels followed 
by achievability. 

Definition 2: Let D/(x) C D(x) be the collection of 
distributions pvsxY on x 5 x A" x 3^ where V is a finite 
field. For pvxsY G ^/(x), let /9(pvxsy) be defined as the 
set 

i?2) :i?i+i?2 <min {^i\)J||^)j- H{V, ® V2\Y) } . 

Let /3('r) be defined as the convex closure of the union of 
P{pvxsy) over pvxsv e D/(t). 
Theorem 2: I3(t) C C{t). 

^This is also because the channel is noiseless. 

^The technique proposed by Philosof and Zamir is particular to the binary 
doubly dirty MAC - Hamming cost constraint and additive nature of the 
channel. Moreover, linear codes only achieve the symmetric capacity, and 
therefore if the output were obtained by passing {X" ©2 ©2 SJ) 

through an asymmetric MAC, linear codes though applicable, might not be 
optimal. 



Before we provide a proof, we state the coding technique and 
indicate achievability of promised rates. 

the key aspect is to employ cosets 



As stated in section III-A 



of a common linear as the source code for quantizing the state. 
We employ three nested linear codes -one each to the two 
encoders and the decoder- that share a common inner (sparser) 
code. We begin by describing the encoding rule. The nested 
linear code provided to encoder j is described through a pair 
of generator matrices gj e y'-x" and Qoi/i G v'^^" where 
91 



(i) gi and goj 



9oj/i 



9oj/i 

are generator matrices of inner 



(sparser) and complete (finer) codes respectively, (ii) 



'>1 

n 



k + li+h 



min{g(yi|^i),g(V2|^2)} 
logTr 

H{Vi®V2\Y) 



(1) 



< 1 - 



logTT 



with TT : = |V|. Let A/ and Aoj denote 
corresponding to generator matrices gi and 
tively. User j''s '^■^^^""■^ ",^'3 r- ^■>l 

(a^gi ® 



message M-- 



(2) 

linear codes 
gO] respec- 
£ V'j indexes the coset 

J 9oj/i ■ fl" G j. Encoder j observes state 5" 
and looks for a codeword in the coset indexed by the message 
that is jointly typical with the state sequence 5" according to 
Ps-Vy If it finds one such codeword, say Vp, a vector X" is 
generated according ]Xi=iPx,\s,Vj{-\SjtVjt) and X"^ is input 
on the channel. Otherwise, it declares an error. We remark 
that the encoding rule is identical to the unstructured coding 
technique except for the source codes being cosets of a linear 
code. 

Now to the decoding rule. The decoder is provided with 
the nested linear code described through the pair of generator 



matrices gj e y'^x"^ go/i • = 



901/I 
902/ 1 



where gj and go/i 



are generator matrices of inner (sparser) and complete (finer) 
codes respectively. Let Ao denote the complete code provided 
to decoder, i.e., the linear code whose generator matrix is go ■ 

. Having received F", it lists all codewords in Ao 

_ 9o// J 

that are jointly typical with with respect to pviBV2,Y- If 
all such codewords belong to a unique coset (of A/ in Ao) 
say {a'^gi ® m'^'goi// © m'^902/1 ■ a'' & V"), it declares 
{rn^,rn2) as the pair of decoded messages. Otherwise, i.e., 
it finds codewords in different cosets jointly typical with F", 
it declares an error. 

As is typical in information theory, we upper bound the 
probability of error event by averaging over the ensemble of 
codes, in this case nested linear codes. For the purpose of 
proof, we consider three nested coset codes instead of nested 
linear codes and average uniformly over the entire ensemble of 
nested coset codes. In particular, we pick entries of each of the 
constituent generator matrices 91, goi/ii 902/i independently 
and uniformly from V. Lower bound ([TJ enable us drive 
down the probability of encoder not finding a jointly typical 
codeword in the indexed coset. This bound can be interpreted 
easily. If we picked codewords according to HtLiPv', then 
we need the bin to be of rate H{Vi) — H{Vi\Si). Since we 



average uniformly over the ensemble of linear codes, each 
codeword of a linear code is uniformly distributed over V". 
Hence the bin must of rate at least logTr — _ff(Vi|5'i). The 
decoder makes an error with arbitrarily small probability if 
Q is satisfied. This bound can also be interpreted intuitively. 
If the codewords were picked according to Pvievb, the upper 
bound would have been H{Vi ® V2) - H{Vi ® V2\Y). In this 
case, the codewords in the sum of nested linear codes are also 
uniformly distributed over V", and this explains the bound in 
0. From (fT|, (B it can be verified that Ri + R2 = < 
min{H{Vi\SiJJIiV2\S2) - H{Vi © V2\Y)} is achievable. 

We emphasize that joint typical encoding and decoding 
enables us decode the sum over an arbitrary MAC-DSTx. 
Indeed, the analysis of joint typical encoding and decoding 
with correlated codebooks (users 1 and 2 employ cosets 
of a the same linear code to partition their channel codes) 
involves new elements. The informed reader will recognize 
the need to prove statistical independence of a codeword in a 
competing sum coset and the pair of cosets indexed by the 
messages. The dependence built across the codewords and 
cosets as a consequence of the algebraic structure exemplifies 
the interplay of algebra and probability. The following proof 
details these elements. 

Proof: We begin with a brief review of coset and nested 
coset codes. An (n, k) coset is a collection of vectors in V" 
obtained by adding a constant bias vector to a /c— dimensional 
subspace of V". If Xq Q V" and A/ C Xq are {n, k + I) 
and {n, k) coset codes respectively, then cosets Xq/Xi that 
partition Xq is a nested coset code. 

A couple of remarks are in order. An {n, k) coset code is 
specified by a bias vector 6" e V" and a generator matrix g e 
V'^''". If Xo C V" and A/ C Xq are (n, k + l) and (n, k) coset 
codes respectively, then there exists a bias vector 6" G V" 

and a generator matrices gj e V*^^" and go — 

I 9o/i 

y{k+i)xn^ such that 6", gi specify A/ and go specify Aq. 
Therefore, a nested coset code is specified by a bias vector fo" 
and any two of the three generator matrices gj, go/i and go- 
We refer to this as nested coset code (n, k, I, gi , go/i^b"^)- 

Our coding technique is based on nested coset codes. User 
j is provided with a nested coset code {n,k,lj,gi,goj/i,b") 
denoted Xoj/Xi. User j's message is used to index one among 



cosets of Xoj/Xj, where tt 



assume message set is V'j and M, 
distributed random variable 
We let v^j{a^,rn^) ■ — 
generic codeword in Xoj/Xi and Cj{rn,j) 



\V\. For simplicity, we 
^ e V'^ to be uniformly 
representing user j's message. 



a'^gi © goj/i © denote a 



g V'^) denote coset corresponding to message m^' . 
We now specify encoding rule. Encoder j observes state 
sequence 5" and declares error if S*" ^ T^{Wsj)- Else, 

it looks for codewords u" (a* 

{S„vj{a\M'^)) e ^[ps,v,\ 

codeword, one of them, say w"(a'^, m'^), is chosen uniformly 





e c.j{Mh, such that 



If it finds at least one such 



to the channel, where f : V x Sj ^ Xj is a map such 
that px^\s,v,{f{sj,Vj)\sj,Vj) = 1. Otherwise, an error is 
declared. 

We now specify the decoding rule. The decoder is pro- 
vided with nested coset code {n,k,l,gi,go/i,b'"') denoted 



Ao/A/ where I ^ h + h, go/i 



901/I 

902/I 



and 



a'^gi © m^go/i ® ^" 
e V'^) denote a generic 



6" : = 6" © 62. With a slight abuse of notation, we let 
m' : = {rn-l ,1112) S : = V'^ x V'^ represent a pair of 
messages and analogously Af' : = [M^^ , Afj^) denote random 
variables corresponding to the users messages. For e 
and m} G V', let v^{a^,m'-) : 
and c{m}) : ~ {v"-{a^ ,m}) : a' 
codeword in coset corresponding to message pair m} and coset 
corresponding to message pair m' respectively. The decoder 
receives y" and declares error if y" ^ Ts{py)- Else, it Usts 
all codewords u"(a*'', m') g Xo such that (u"(a*'', m'), y") G 
Tg^{pVi(BV2,Y)- If it finds all such codewords in a unique 
coset say c(m') of Ao/A/, then it declares rh' as the pair 
of decoded messages. Otherwise, it declares an error. 

As is typical in information theory, we derive an upper 
bound on probability of error by averaging the error probability 
over the ensemble of nested coset codes. We average over the 
ensemble of nested coset codes by letting the bias vectors : 
j — 1,2 and generator matrices G/, Goj/i : j = 1, 2 mutually 
independent and uniformly distributed over their respective 
range spaces. Let Aoj/A/ : j — 1,2 and Ao/A/ denote the 
random nested coset codes (n, fc, Ij, Gi,Goj/i, BJ) : j — 1,2 
and {n,k,l,Gi,Go/i;B"-) respectively. For a'' G V*^, m' S 
V', let Vp{a'',m^) : j = 1,2, y"(a'=,m') denote corre- 
sponding random codewords in Aoj/A/ : j = 1, 2 and Ao/A/ 
respectively. Let Cj{rnj') : = {VJ^ (a'^ , nij' ) : a'^ £ V^) and 
C{m}) : = (T^"(a'^, m') : <E V^) denote random cosets 
in Aoj/A^i ■ j ~ 1,2 and Ao/A/ corresponding to message 
irij : j = 1,2 and m' respectively. We now analyze error 
events and upper bound probability of error. 

We begin by characterizing error events at encoder. If eji : 
= {S'^^Ts{Ws^)}, 



, m. 



E 1 



V^"(a^m^^),s5')eTJ(pv_^s 



and €j2 '■ — {4>{S''-' , M ) — 0}, then eji U £^2 is the error 
event at encoder j. By frequency typicality, P{eji) < | for 
sufficiently large n. An upper bound on P{€j2 H e^^) can be 
derived by following the arguments in ||3]. Findings in |[3l 
imply existence of Nj2 G N such that Vn > Nj2, P{£2j H 
> 1 " 



H(V,\S,) 
"JJ-'' — 8 " n ' " logTT ■ 

The error event at the decoder is e^Ue^, where £3 : — {Y" ^ 

T^Py)} and 



at random and ej{M/,S^) : = /j*(Wj"(a'^, Mj' ), S"]') is input that Vn > N3, F((u2^i£ji U £^2)'= n £3) < |. It remains to 



^4:= U U {(^"(«''^')'^") e^5"(py.ey..y)}- 
By conditional frequency typicality, there exists N3 £ N such 
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Fig. 1. Bounds on sum rate for example [T] 



upper bound P(((u|^jeji U 6^2) U n £4). In appendix [a| 
we prove that if ^ < 1 - there exists e N 

such that Vn > N4, P{ei) < |. ■ 

C. Examples 

We now illustrate significance of theorem [3] through the 
following non-additive examples. A few remarks are in order 
The examples needing to be non-additive lends it considerably 
hard to provide analytical upper bounds for the rate region 
achievable using unstructured codes|^ We therefore resort to 
computation. Computation of a{T) involves two auxiliary al- 
phabets of cardinality at least 3 each just for the binary MAC- 
DSTx. Sampling the probability matrix with any reasonable 
step size is infeasible. It is our conjecture that the additional 
letter in the auxiliary alphabet enables convexification of the 
rate regionj^ The sum rate bound for the unstructured coding 
technique projected below is therefore obtained through com- 
putation involving binary auxiliary alphabet sets followed by 
convexification (timesharing between different cost). Secondly, 
one will expect the gains projected in |2| to carry over for 
channels close to additive. The following examples confirm 
that. It is therefore reasonable to believe that restriction of the 
size of the auxiliary alphabet is not altering the results. The 
following examples being non-additive lends PZ-technique 
inapplicable. For the following examples, we assume alphabet 
sets to be binary Sj — Xj = {0, 1}, j = 1, 2, (ii) uniform and 
independent states, i.e., Ws{s) = | for all s G 5, (iii) a 
Hamming cost function Hij{l,Sj) = 1 and Kj(0,Sj) — for 
any Sj £ Sj, j — 1,2. We resort to computing upper bound 
achievable using unstructured codes and sum rate achievable 
using nested coset codes. These are plotted in figures T]|4 



Example 1: The channel transition matrix is given in table 
|l] 1) An upper bound on sum rate achievable using unstruc- 
tured codes and 2) sum rate achievable using structured are 
plotted in figure [T] 

Example 2: The channel transition matrix is given in table 
[n] 1) An upper bound on sum rate achievable using unstruc- 

'We recognize that tlie analytical upper bound derived in (2J is a key 
element of the findings therein. 



52 X2S\Xi 




S2X251X1 


Wyisjf (0|.) 


0000 


0.93 


1000 


0.08 


0001 


0.04 


1001 


0.84 


0010 


0.06 


1010 


0.86 


0011 


0.89 


1011 


0.11 


0100 


0.11 


1100 


0.87 


0101 


0.81 


1101 


0.08 


Olio 


0.79 


1110 


0.09 


Dili 


0.12 


1111 


0.91 



TABLE I 

Channel transition matrix Example[T| 
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Fig. 2. Bounds on sum rate for example |2] 



tured codes and 2) sum rate achievable using structured are 
plotted in figure |2] 

Example 3: The channel transition matrix is given in table 
III 1) An upper bound on sum rate achievable using unstruc- 
tured codes and 2) sum rate achievable using structured are 
plotted in figure [3] 

Example 4: The channel transition matrix is given in table 



$2 X2S1X1 


»V|sx(ol-) 


S2X2S1X1 




0000 


0.90 


1000 


0.03 


0001 


0.04 


1001 


0.89 


0010 


0.06 


1010 


0.89 


0011 


0.83 


1011 


0.06 


0100 


0.08 


1100 


0.95 


0101 


0.83 


1101 


0.03 


Olio 


0.89 


1110 


0.05 


0111 


0.10 


1111 


0.94 



TABLE II 

Channel transition matrix Example[2] 



S2 X2SIXI 


"V|sx(o|-) 


S2X2S1X1 




0000 


0.92 


1000 


0.07 


0001 


0.08 


1001 


0.92 


0010 


0.06 


1010 


0.96 


0011 


0.94 


1011 


0.10 


0100 


0.10 


1100 


0.88 


0101 


0.92 


1101 


0.08 


Olio 


0.95 


1110 


0.11 


0111 


0.06 


1111 


0.91 



lOi 



The optimal test channel for the binary doubly dirty MAC does not 
contradict this. 



TABLE III 

Channel transition matrix Example[3] 
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Fig. 3. Bounds on sum rate for example [3] 



bound on sum rate achievable using unstructured codes 

sum rate achievable using nested coset codes 
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Fig. 4. Bounds on sum rate for example |4] 



IV 1) An upper bound on sum rate achievable using unstruc- 
tured codes and 2) sum rate achievable using structured are 
plotted in figure [4] 

In computing above bounds, we restricted (i) auxiliary 
alphabet sets Ui , U2 to be of cardinality at most 2, (ii) Xj to 
be a function of Uj , Sj p] The resulting space of probability 
distributions that respect the cost constraints was sampled with 
a step size of 0.015 in each dimension. The resulting bound on 
the sum rate achievable using unstructured codes (without time 
sharing) is marked with blue crosses (denoted a in the legend) 
in the plots. More formally, for a cost constraint t — (r, r). 



"Since the objective function I(U; Y) + I(Ui; l/2)-E^=i HUj;Sj) is 
(i) convex in Pxi\UiSi for a fixed P(7i j Si , P(72 1 S2 ' P-Jf 2 1 1^2 S2 ' ("> convex 
in PX2 1 C/2 S2 f™ a ^^^'^ Pui\SiyPU2\S2'PXi\UiSi this restriction is valid. 





"'risjc(ol-) 


S2X2S1X1 


"'y|sx(0|-) 


0000 


0.93 


1000 


0.08 


0001 


0.04 


1001 


0.84 


0010 


0.06 


1010 


0.86 


0011 


0.89 


1011 


0.11 


0100 


0.11 


1100 


0.87 


0101 


0.81 


1101 


0.08 


Olio 


0.79 


1110 


0.09 


0111 


0.12 


nil 


0.91 



TABLE IV 

Channel transition matrix Example|4] 



a is representative of 

2 

sup_ I{U;Y)+I{Ui;U2)-J2l{Uy,S,) 

Pusxy&D{t) j — i 

where D(x) C D(t) is the collection of distributions defined 
on U^xSxXxy where (i) \U\ < 2 and the channel transition 
probability matrix is as provided in the corresponding table. 
The resulting upper bound is obtained as an upper convex 
envelope. Similarly, sum rate achievable using nested coset 
codes is marked with red circles (denoted (3 in the legend) in 
the plots. More formally, for a cost constraint r = (r, r), (3 
is representative of 

sup_ mm{H{Ui\Si),H{U2\S2)-H{Ui®U2\Y)} 

PUSXY&Df{T) 

where Dj(t) C D(t) is the collection of distributions defined 
on U^xSxXxy where (i) \U\ < 2 and the channel transition 
probability matrix is as provided in the corresponding table. 

We conclude with remarks on the choice of examples. The 
channels studied in the examples are obtained by slightly 
perturbing an additive channel and are therefore 'close to 
being additive'. The reader may question this choice. Since 
theorem |3] characterizes the rate region achievable by decoding 
the sum of auxiliary codewords, we do not expect this to 
outperform the technique of decoding the pair of codewords 
unless the effective channel induced between the auxiliary 
inputs (C/i, U2) and output Y is not far from being additive. 
As mentioned earlier, techniques based on structured codes 
are more efficient only if the function decoded is compressive 
enough to offset the loss incurred for imposing structure in the 
codebooks. For the problem in context, the sum rate achievable 
using structured codes is H{Ui\Si)+H{U2\S2)~H{UiU2\Y), 
and therefore, we expect gains if H{Ui(S)U2\Y) is sufficiently 
smaller when compared to H{UiU2\Y). 

Does this imply the applicability of structured codes only 
to problems that are close to being additive. The findings of 
theorem |3] are only hinting at potential gains achievable by 
decoding bivariate functions through the simplest example of 
field addition. Indeed, a comprehensive technique will involve 
choosing the appropriate bivariate function for the problem 
in context and building codes whose source codes are closed 
with respect to that bivariate function. We elaborate on this in 
section [V] For the case of field addition, the example provided 
by Philosof and Zamir |2| is indeed canonical and the exam- 
ples provided herein, provided for the sake of completeness, 
only corroborates the phenomenon at play. 

We conclude by stating that the rate region achievable using 
nested coset codes for the above examples can be potentially 
enhanced by incorporating unstructured code techniques. In 
other words, we build on pl, to propose a universal strategy 
that subsumes both unstructured and structured coding tech- 
niques. This provides further credence for potential enlarge- 
ment of achievable rate regions using structured codes. 



IV. A UNIFIED ACHIEVABLE RATE REGION 

In this section, we put together the techniques of unstruc- 
tured and structured random coding to derive the largest 
known achievable rate region for a general MAC-DSTx. Our 
approach is similar to that proposed by Ahlswede and Han |4 
Section VI] for the problem of reconstructing mod— 2 sum of 
distributed binary sources. We begin with a characterization 
of valid test channels. 

Definition 3: Let Ds/(t) C D(t) be the collection of 
distributions puvsxY on {U x x S x X x y where U 
is a finite set and V is a finite field. For puvsxY G ^^/(t), 
let TZipuvxsY) be defined as 

' i?2) : < i?i < /(C/i; U2Y) - liUi; Si) 
+ mm{HiV,\U,,Sj) : j = 1, 2} - HiV^ ® \^2|C/i, f/2, Y) 
< R2 < IiU2;UiY) - I{U2; S2) 
' +min{H{Vj\Uj,S/}:j = l,2}-H{Vi®V2\Ui,U2,Y) ' 

Ri + R2< IiUiU2; Y) + I{Ui-U2) - I{U,\Sj) 
^ +uim{H{Vj\U,,Sj) : 3 = 1,2} - H{Vi ® V^2|C/i, f/2, Y) , 

where © is addition in V. Let 7?.(r) be defined as the convex 
closure of the union of TI{puvxsy) over the collection 
PuvxSY S ^sfiT) of valid test channels. 
Theorem 3: 72.(x) C C{t). 

In the interest of brevity, we briefly state the coding tech- 
nique. Each user builds an unstructured code over U and 
a nested linear code over V The nested linear codes share 
a common inner (sparser) code. The unstructured code is 
partitioned into bins (Gelfand-Pinsker binning). Each user's 
message is split into two parts - one indexing a bin in the 
unstructured code and one indexing a coset in the nested linear 
code. Encoder j picks a pair of codewords, say (t/",Vj") 
jointly typical with the observed state sequence from the 
indexed pair of bins and transmits an input vector generated 
according to PXj\VjUjSj - Having received Y", decoder looks 
for all triples (C/f , U2, V" ® V2") of codewords in the corre- 
sponding codebooks that is jointly typical with according 
to Pu,Vi®V2,Y ™d declares the quadruple of bin indices as the 
decoded message. Achievability is proved by upper bounding 
the probability of error averaged over the ensemble of codes. 

Remark 1: q;(t) C TI{t) and moreover TZ{t) is the largest 
known achievable rate region for the general MAC-DSTx. 

V. Enlarging achievable rate region using codes 
OVER groups 

Consider a quaternary doubly dirty MAC-DSTx (QDD- 
MAC), with Sj = = y = {0,1,2,3}, j = 1,2. The 
state sequences are independent and uniformly distributed, i.e., 
l^s(s) — for all s £ S. The channel transition is described 
by the relation Y = Xi ^ Si ^ X2 ^ S2, where ^ denotes 
addition mod— 4. All nonzero symbols have equal cost, i.e., 
Kj{x,Sj) = 1 for all X € {1,2,3} and Kj{0,Sj) = for all 
Sj £ Sj, j = 1,2 and the input is subject to a symmetric cost 
constraint r = (t, r). 

What would be the achievable rate region for QDD-MAC 
using unstructured codes? The technique proposed in |2| that 
yields an upper bound on the sum rate achievable using 




Fig. 5. Sum rate achievable using unstructured, nested coset and group codes 
for test channel j3j 

unstructured code for BDD-MAC cannot be eniployed to 
derive an analogous upper bound for QDD-MACPj We are 
led to conjecture the form of the optimizing test channel takes 
the form 

, . , I ^ for = 

Px, u, I s, [Xj ,Xj ^ Sj\sj) = i^^^ otherwise 

The conjecture is based on empirical evidence obtained 
for the corresponding quaternary dirty point to point 
Gelfand-Pinsker channel. The sum rate achievable using 
the above test channel is the upper convex envelope of 
the function a : [0, |] -> [0, 00) defined as a(T) = 
max{-2Tlog(f)-2(l-T)log(l-T))-2,0}. Since 4 is 
a prime power, there exists a unique field of cardinality 
4. Do nested coset codes built over J^j achieve a larger sum 
rate? 

We are unable to characterize sum rate achievable using 
nested coset codes and the dimensionality of the space of prob- 
ability distributions lends computation infeasible. We conjec- 
ture the above test channel optimizes the sum rate achievable 
using nested coset codes. In any case computing the sum rate 
achievable using nested coset codes for the above test channel 
is instructive. It can be verified that the sum rate achievable us- 
ing the above test channel with nested coset codes is the upper 
convex envelope of the function /3 : [0, |] — )■ [0, 00) defined 
as /3(t) = max {-Tlog(f ) - (1 - r) log(l - r)) - i, O}. 

The sum rate achievable for the above test channel using 
unstructured and nested coset codes is plotted in figure [5] It 
is no surprise that nested coset codes perform poorly. The 
channel operation is not the field addition 04 in 7^4. Instead, 
^ is the group addition in the Abelian group Z4. This suggests 
we build codes over groups that are closed under group 
operation and decode the sum of auxiliary inputs. 

Linear codes are kernels of field homomorphisms that are 
also known as linear transformations. This lends them the 
property of closure under field addition. We build group codes 
that are kernels of group homomorphisms. Group codes are 
closed under group operation. We choose a group code Tj. 

'^In fact, this technique cannot be generalized to an arbitrary finite field. 
For this case, Wyner's technique fl7' Section II] seems more promising. 



Cosets of F/ in a larger group code Tqj, that contains Tj, 
is chosen to be user j's code. Cosets of Tj form bins of 
each user's codebook. The encoding is similar to that with 
nested coset codes. The decoder decodes group addition ^ of 
auxiliary codewords. 

What are the rates achievable using group codes? In 
the interest of brevity, we only provide a sampling of 
results for QDD-MAC. We refer the interested reader to 
1181 for a study of group codes and their potential in 
multi-terminal communication systems. Just as linearity 
forced us to enlarge rate of a bin from I{Uj;Sj) to 
log|Z/,| — H{Uj\Sj), closure under group operation will 
entail an analogous rate loss. The algebraic structure being 
more degenerate, the rate loss is in general larger Yet, for 
the QDD-MAC, the sum rate achievable using group codes 
for the test channel in [3] is larger In particular, the sum rate 
achievable using group codes is the upper convex envelope 
of the function (3 : [0, |] [0, oo) defined as I3{t 
max {min {-r log(f ) - (1 - r) log(l - r), 2/ib(f )} , O} 
In figure |5] the sum rate achievable using group codes for the 
above test channel is plotted. 

The results in this section, though incomplete, point to a rich 
theory of strategies for multi-terminal communication systems 
based on structured code ensembles. Gains crucially rely on 
the compressive nature of the bivariate function and the ability 
to build efficient codes with rich algebraic structure. It is 
therefore no surprise that all of earlier findings were based on 
exploiting modulo— 2 sum - the simplest compressive function 
with binary arguments - using linear codes - an ensemble that 
has been studied at length from different perspectives. 

Appendix A 
An upper bound on P(e4) 

In this appendix, we derive an upper bound on P{e4). 
As is typical in proofs of channel coding theorems, this 
step involves establishing statistical independence of cosets 
Cj{Mj^ ) : j = 1,2 corresponding to the message pair and any 
codeword y (a'^, m') in a competing sum coset. We establish 
this in lemma |2] We begin with the necessary spadework. 
Throughout this appendix, we employ the notation introduced 
in proof of theorem [3] 

Lemma 1: If m} ^ m} , then for any triple i/",z^2:^" € 
V", 



where to ■ 



G TO . We now prove, using a counting 



p I V/(0^m/)=Iy7:i = 1.2, 
y"(0'',rri') = i>" 



1^=1.7:^ = 1,2 J \ =0" 



Proof: By definition of Vj {O'' , rn^^ ) : j — 1,2 and 
y(0'=,m'), 

V"(0^rrl') = l>" 



argument, the term on right hand side of (jiji is ^p^*^ Since 
to' 7^ to', there exists t e [I] such that rht 7^ to^. Given any 
{I — 1) vectors gQ jj j G V" : G {1, • • • , Z} \ {i}, there exists 
a unique triple of vectors {go //.t, , ) ^ ^ ^ such 



that 



to'/ O'^ 



go/i®h'l = 



9o/i 



Qh 



9o/i®b'^ = and 



Hence 



go/r- 



= TT^'^^)". 



The mutual independence and uniform distribution of 
Go/ 1, Bi, B2 implies the term on RHS of ^ is indeed 
It remains to prove 

lope 

\ = P(^^"(0^m^O = : J = l,2)P(y"(0^m') - />") = — . 

Note that codewords of a uniform random linear code are 
uniformly distributed i.e., P (y"(0'^, rn,') = i)") = Using 
the definition of y"(0'^,m'), we only need to prove 



P 



0'-i^Go/i<SB^=L'i, 
[o'l ml,''] Go/ 1 (BB'^' = 1^2 



1 

^2n ' 



This follows again from a counting argument. For every matrix 
go /I e V'^", there exists a unique pair of vectors 6", 63 G V" 



such that 



0'^ Go/ 1 ® Bl' = and 
B2 = ^2 '^hus yielding 

■(9o/.,fer,fc?)e . K o'^]Go/.ffiBr=. 



o'l 



G 



o/i < 



'xV"xV" 



= 7r('-2)'^ 



(5) 



and the proof is completed using the mutual independence and 
uniform distribution of Go/i, B" , B"!^. ■ 
Lemma 2: For any m' 7^ m', and any a'^ G V*^, the pair 
of cosets Cj{nij') : j = 1,2 is statistically independent of 
y"(a'=,m'). 

Proof: For j = 1, 2, let ^-"(4) G V" for each a'^ G V'=, 
and z>" G V". We need to prove 

P(C;(TO;^) = (i/,(4):4GV'=):j = l,2, 
y"(a^Tf^') = z)") 
= P(q'(TO^) = (^(4):4GV'=):j = l,2) 
P(F"(a^»T^') = i>") 

for every choice of Vj{a^) G V" : G V^^j = 1,2 and 

If (i) for j = 1 or J 2, {vj{a^^®h^j)Qv,{Q*')) ^ {iyj{a'^)e 
i'j(0'=)) ® {i^j(a^) e i^jiO'')) for any pair a'^, a/ G V*^, or 
(ii) 1^1(05^) e ui(0'') 7^ i^2(a?) e 1^2(0'') for some G V'', 



^Recall 7r : 



|V|. 



■*row j oi go/i is Qo/lJ 



then LHS and first term of RHS are zero and equality holds. 
Otherwise, 



We are now equipped to derive an upper bound on -P(e4). 
Observe that 



P {cJ{ni'/) = iu,ia']y.a';€V''):3 = 1.2,V"{&'',rh')=0") 

/ 4G^ = ^/2(a'=)eI/2(0'°):4eV^ \ 
= P Vpi0'',m'/)='^j{0''):j = l,2., 

\v"(o\rri')=i>"e(i^2(a'')e>.2(o'=))/ 

y/(0\m^^)=i.,(0'=):j = l,2. 

y " (0'= .rfi' ) " e(i^2 (a'° )ei'2 (0*= 



P(e4) < P 



(6) 



U U {'^^'' 



m'),y")GT5(pv-^ev2,y) 
M'=m' 



} 



m ,m 



p /ajGi=i/2(a'=)e\ p 



^ E E E E 



(7) 



Mm' eT5(F)T25(yieV2|!y") 



P 



''o2G/2=''2(a'°)ef2(0'=):a2GV 

i'/ 0'2]Go//eB;'=i/i(0'=), j p |'v"(a'=,rri') 



P (G7(m;^) = (.,(4):aJeV'=):, = l,2) P (^"(f^. 



P(y"(a''\m') = i>") (8) 
(9) 
(10) 



^ E E E T.^r^n^'i^^ 



'im' T^(y)T2a(yiffiy2|y") 



^ E E E E 

a'=GV'=m'6V' J/ 



GTi(y) T25(VieV2|y") 



< 



E 



^''+'\T2s{Vi®V2\y^)\ 



y 



where i) JSb and ( 10 



follow from definition of cosets Cj {m- ), < gxp < — n log tt ( 1 — 



(ii) (|7]i and (|9]) follow from independence of Gi^ and the 
collection [Go/itB^I^B!^) and (iii) (|6]l follows from lemma 

m ^ ■ 

We emphasize consequence of lemma |2] in the following 
remark. 

Remark 2: If m} ^ to', then conditioned on event 
{M' = to'}, received vector is statistically independent 
of V"{a'^ ,rn'') for any a'' G V*^. We establish ti-uth of this 
statement in the sequel. Let Cj denote the set of all ordered 
Tr'^'-tuples of vectors in V". Observe that 



H{Vi®V2\Y) + 3S _ k + l 
log TT n 



(12) 



where ( [T2] i follows from the uniform bound of 

e^^{n{H{Vi(BV2\Y) + i5)) on \T2s{Vi © ^sly")! for 
any S T5(y), for sufficiently large n (bound on 



/ M'=rn^ 



m' )=&" 



E E 

CiGCi C2eC2 s"e5 



E ^ 



^J\/'=m',5"=s"^ 
Ci(m'ii)=Ci 
C2(m;^2)^C'2 



C2(m^2)^C'2 



CiGCi C2GC2 s"e5" 
P(^/"(a^m')=^)")P ( F" y« | C, (m/ )=C, :j = l,2 | ^^^^ 



M'=m',Y'^=y",S"=s' 



I C,(m'/)=Cj:j = 1.2 

CisCi C2eC2 s"e5" ^ 
P (M' = to', r" = y'^) P (t/"(a'=, m') = i>") 



p I V"(d'',m') 



where ([TT} follows from (i) independence of random objects 
that characterize codebook and (S*", Af '), (ii) lemma|2]and (iii) 
statistical independence of the inputs Xj{Mj^ ' ^f) '-J ~ ^■'^ 
to the channel and the codeword V" (d'^ , to ) conditioned on 
the specific realization of cosets {Cj{M^^') : j = 1, 2) and the 
event {Af' = to'}. Moreover, since P(T^"(d'^, ?fi') = i>") = 
^, we have P (M' = to', F" = y", P""(d'=, m') = i>") = 
%P{M^ = m\Y" = y"). 



conditional frequency typical set). (12 1 is the desired upper 
bound on P(e5). 
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